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Introduction 


The purpose of this unit is to examine one of the principal methods for solving 
sets of equations such as 


1.2x, + 0.3x, — 1.3x, = 12 E 
3.3x, — 11x, + 2.4x, = 5.3 £. 
4.2x, + 2.3x, + 7.3x3 = 7.4 E. 


These are called simultaneous algebraic linear equations. In this instance we have 
three equations, where the problem is to determine values for the unknown 
variables x,, x, and x; so that all three equations are satisfied simultaneously. 
They are called algebraic as their solutions are numbers (as compared with the 
sequences which were the solutions to the recurrence relations in Unit 1, or the 
functions which were the solutions to the differential equations in Units 2 and 6). 
Finally, the term linear derives from the case of two simultaneous equations in two 
unknowns, where each equation can be represented by a straight line. 


For brevity, these sets of equations will just be referred to as simultaneous 
equations, and the individual equations as E,, E,, etc. 


Equations of this form occur frequently in practice. The unknowns could be 
currents in an electrical network, for instance, or stresses in an engineering 
structure, or quantities of different goods transported from place to place. In such 
applications, we might be dealing with hundreds, or even thousands, of equations, 
in which case a computer is essential. 


Simultaneous linear equations also occur frequently as steps in the solution of 
other kinds of mathematical problem: for example, selecting a particular solution 
of a recurrence relation or a differential equation by using initial conditions. 


The discussion in this unit is largely confined to two equations in two unknowns, 
or three equations in three unknowns. However, we shall see that the method of 
solution chosen can easily be extended to larger sets of equations. There are many 
methods for solving simultaneous equations. We have chosen a particular version 
of the method called Gaussian elimination, for the following reasons: 


(i) because of its simplicity, 


(ii) because it is systematic and can easily be carried out on a computer (which is 
essential if it is to be of any practical use), 


(iii) because it clarifies the different types of solution that arise, and the various 
things that can go wrong. 


Study guide 


You should study Sections 1 and 2 and Subsection 3.1 before watching the 
television programme. 


There are several ways of setting out Gaussian elimination, so even if you have 
met the method before, please read Section 1 to ensure that you can follow the 
way it is done in this unit. 


Section 4 describes a computer package which will enable you to solve systems of 
linear equations by Gaussian elimination. One of the assignment questions may 
require you to use this package. We suggest that you combine this terminal visit 
with the one for Unit 10. Before making this visit, you should have studied 
Sections 1 to 4 of Unit 9 and Sections 1 and 2 of Unit 10. 


The exercises at the end of each section are designed to give extra practice. It is 
not necessary to do all of them when you study the unit; indeed, you may wish to 
save some of them for revision purposes. However, you are encouraged to do the 
end of unit test immediately after studying the unit. 


The five sections of this unit are not of equal length. You will probably find that 
Sections 1 and 5 each take you only about half as long as the other sections. 


x 


solution 


x 


Units 1, 6 


Sp 
==. 


Carl Friedrich Gauss 
(1777-1855) was described by 
Laplace as ‘the greatest 
mathematician in the world’. 
He made a large number of 
major contributions to 
mathematics and physical 
science. Amongst other things 
he devised the method of 
successive elimination for 
solving sets of simultaneous 
equations, a version of which 
is given in this unit. 
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1 Linear simultaneous algebraic equations 
and their solution 


1.1 A ‘practical’? example 


John, who is slimming, is planning a simple breakfast of cereal and sugar 
substitute. John is an eccentric character who drinks water with his breakfast, and 
eats his cereal dry. He has decided to allow himself exactly 500 calories and 10 
grams of protein. The calorie and protein content of the foods are shown in the 
following table. 


Sugar 
substitute 


Cereal 


Calories 
per gram 


Grams of protein 
per gram of food 


He would like to know how many grams of cereal and of sugar substitute he 
should eat to satisfy his requirements. 


Let the required amount of cereal be x, grams, and the amount of sugar substitute 
be x, grams. Then the total calorie content of the meal is 


6x, + 2x, calories 
and the protein content is 
O.1x, + 0.1x, grams. 
So if the requirements are to be met, we must have 
6% 4: 2x, = 500 
and O.1x, + 0.1x, = 10. 


We have to find the values of x, and x, which satisfy these equations 
simultaneously. 


A systematic method for solving equations of this type will be described in 
Subsection 1.2. You have already had to solve pairs of equations like this in 
previous units, and should have no trouble in arriving at the solution 


x, = 75, x, = 25. 


So John must eat 75 grams of cereal and 25 grams of sugar substitute to satisfy his 
requirements. 


In more realistic problems the number of unknowns and of equations can easily be 
larger than 2. For example, if John had also had toast for breakfast, we should 
have brought a third unknown x, into the equations. Had he taken Vitamin C 
into account as well we should have got a third equation. 


The rest of this section is concerned with developing a systematic method for 
solving sets of simultaneous equations, starting with the case of two equations in 
two unknowns. 
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1.2 Solution of two simultaneous equations in two unknowns 


We should like to develop a systematic method for finding the values of x, and x, 
which satisfy the equations 


AX, + b,x, = c, aS 
where the as, bs and cs are given quantities. 
Simultaneous equations have one very useful property, which we use time and time 
again. If we add or subtract any multiple of one equation to or from another, then 
the values of x, and x, which satisfy the original equations will satisfy the new 
one as well. (If you have studied M203, you may remember seeing a proof of this.) M203, Unit 11 


We use this property to obtain a new equation with no term in the first unknown 
Le 


Example 1 


Subtract a multiple of E, from E, to obtain an equation which has no term in x, 
from the simultaneous equations 


x, + 2x, =5 E, 
2x, + 7x, = 16. | ‘oe 
If we double the first equation, we get 
2x, + 4x, = 10. 2E, These new equations are 
; labelled 2E, and E, — 2E, 
If we now subtract this from E,, we get the equation because that is how they are 


obtained from E, and E,. 


The advantage of this new equation is that it is simpler than the ones we started 
with. If we now choose to solve the simultaneous equations 


x, + 2x, =5 | Ey 
3%, eS Ey, 


(i) we have a much simpler task, 


(ii) the solution will be the same as if we had solved E, and E,. 


We shall call this new equation E,., as for solution purposes it replaces E,. 


Example 2 
Solve the simultaneous equations of Example 1: 
xX, + 2x, =5 E, 
2x, + 7x, = 16. E, 


This is a two-stage process. 


Stage 1 is the process described in Example 1. The process of subtracting a 
multiple of E, from E, to obtain an equation E,, without a term in x, is called 
the elimination stage. | 


Stage 2 consists of solving the simplified set of equations found in Stage 1: 
Xx; + 2x, =5 Ey 
3x, = 6. E.. 
Now E,, is just a simple equation in one unknown, which we solve to get 
Xo = Z. 
Substituting this value into E,, we get 
X,+4=5 
so that x, = 1. This gives us the solution 


x, = 1, ¥5 m2: 
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Stage 2 is called back substitution as, to obtain the solution, we always start with 
the bottom equation and work back through the equations, substituting the values 
we have found into the equation above, in order to find the next unknown. 


It is always advisable to check your solution by substituting the values that you 
have found back into the original equations. For instance, checking the example 
above, 

LHS of E,: x, + 2x, =1+(2 x 2)=5=RHS. 


LHS of E,: 2x, + 7x, = (2 x 1)+ (7 x 2) = 16 = RHS. 


Exercise 1 
Use the method above to solve the simultaneous equations 


x, -2x,= 5 Ey 
2x, + 3x, = —4 E, 
and check your solution. 


(Solution on p. 53) 


It is not difficult to see how to convert this method into a systematic rule for 
solving any pair of simultaneous linear equations in two unknowns. The general 
form of such a pair is 

aX, + 65x, =c, E, 

a,x, + b,x, =c, E, 
and all we need to do is to decide what multiple of E, to subtract from E, to 
obtain an equation with no term in x). 


If we multiply E, by the multiplier a,/a,, we get the equation 


ay a> 
aX, + |——1b,x, = |—]1¢c,. 
2%1 2), 2), 


1 1 
If we now subtract this from E,, this will provide us with an equation with no 
term in x,. Since we want to do this process automatically, there would be 
difficulties if a, were zero, as we would find ourselves (or the computer) trying to 
divide by zero. You can see the way that we deal with this difficulty in Section 3. 


Rule for obtaining the equation E£,, 
If 


a,x, + b,x, =c, 


A,X; + B3%5 =S 


and a, is not zero, 


a : 
then E, — 2 E, will be an equation without a term in x,, which 
a 


1 
we label E,,. 


This rule should help you with the following example and exercise. 


Example 3 

Solve the simultaneous equations 
2x, + 3x, = 3 E, 
3%, + 2% = =—5 E, 


Stage 1: using the rule above, we know that E, — 3E, will be an equation without 
a term in x,. 


E, —3E, gives us 
(3x, + 2x2) — 3(2x, + 3x2) = —7 — & x 3), 


which simplifies to 


LHS and RHS mean ‘left 
hand side’ and ‘right hand side’. 
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—3x, = —5. E, 
Stage 2: solve the equations 
2X, + 3X = 3 E. 
=—$%, = —5. | Ex, 
From E,,, x, = 2. Substituting this value into E,, we get 


2x, + 3 x 2) = 3. 


Hence x, = —3. This gives a solution 
Xx, = —3, ees 
Check 
LHS of E,: 2x, + 3x, = (2 x (—3)) + G x 2) =3 =RHS. 
LHS of E,: 3x, + 2x, = (3 x (—3)) + 2 x 2) = —4 = RHS. 
Exercise 2 
Solve the simultaneous equations 
3m 2kp = 20 Ey 
ky 8g = —1 E, 


(Solution on p. 53) 


1.3 Solution of three simultaneous equations in three unknowns 


In this subsection, we extend the technique used in Subsection 1.2 to deal with 
three simultaneous equations in three unknowns. As before, the process falls into 
two stages. In the elimination stage, multiples of E, are subtracted from both E, 
and E;. The two new equations formed (E,, and E3,) both have no terms in x, 
and contain only two unknowns., To complete the elimination stage, we treat 
these two equations as before, finishing up with an equation (E3,) in just one 
unknown. 


Example 4 

Solve the simultaneous equations 
2x, + 4x, — 3x, = 16 oF 
3X, — 2x, + 6x, = 13 E; 
8x, — 2x, — 9x, = —2. i 


Stage 1. The elimination stage: 


Stage I(a): first we subtract multiples of E, from E, and E, to get two new 
equations without terms in x. 


E, —3E, gives 

—8x, + 44x, = —-11 ep 
and E, — 8E, gives 

—18x, + 3x, = —66. Es, 


Stage 1(b): we forget about the original equations for the time being. E,, and E3, 
are simply two equations in two unknowns. So, using these two, we subtract a 
multiple of E,, from E3, to obtain an equation E, without a term in x). 
E3, 2 (BE; gives 

ee eae 3p 


At this stage, the elimination process is finished; we have brought our equations to 
what we call the ‘upper triangular’ form: 
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-— 8x, + ay, —_ — 11 es 
= Pig E34 


Stage 2. The back substitution stage: 


We can now find the values of x,, x, and x, from E,, E,, and E,,. Starting with 
E3,, we get 


x, = 2. 
Substituting this value into E,,, we get 
—8x, + (+ x 2) = -11, 
x, = 4. 
Finally, substituting both the values for x, and x, into E,, we get 
ox + (4 xD = 0 % = 16. 
Hence x, = 3, giving the solution 


x, = 3, x, = 4, xX, = 2. 


Checking the solution, we get 

LHS of E,: 2x, + 4x, — 3x, = (2 x 3) + (4x 4) -—(3 x 2)= 16=RHS. 
LHS of E,: 3x, — 2x, + 6x; = (3 x 3) -—(2 x 4)+ (6 x 2)= 13=RHS. 
LHS of E,: 8x, — 2x, — 9x, = (8 x 3) —(2 x 4) — (9 x 2) = —2 = RHS. 


The process of solving simultaneous equations described in Example 4 is often 
referred to as the Gaussian elimination method, although strictly speaking the 
elimination is finished at Stage 1(b). The method, albeit tedious, provides a 
systematic approach to the problem of solving simultaneous equations, which 
should cope with the rather larger sets of equations which can occur in real-life 
applications. 


Exercise 3 
Solve the following simultaneous equations by Gaussian elimination: 


3X5 + XX, = I E; 
3X, +. Xo + 2%, =6 E, 
4x, — 2x, — 3x, =3. E, 


(Solution on p. 53) 


1.4 Organization of the work 
When using Gaussian elimination to solve the equations 


sk eS E; 


the new equation E,, is obtained by subtracting 3E, from E,. It is worth noting 


that the work is done on the numbers multiplying x, and x, (called the coefficients 
of x, and x,). For instance, the operation 


pee | 
2X4 — 3Xq = 3X 
could be recorded as 
2 Ze 
> be eee 


(so long as we remember that the operation was attached to x,!). While the 
elimination is being done, it is sufficient just to record the coefficients of x, and x, 
and the right-hand side of the equations rather than the whole equation each time. 
For instance, we can record the important features of E, and E, above in the 
number block 


Notice that the aim is to get 
successive equations each with 
one less unknown. 
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pa 1 3 
3 4 5. 


The first row contains the coefficients of x, and x, and the right-hand side of E,, 
and the second row contains similar information about E,. Alternatively, the first 
and second columns represent the coefficients of x, and x,, and the column after 
the bar represents the right-hand sides of the equations. 


Often, in mathematics, information is stored in rectangular number blocks in this 
way. A block like this is called a matrix, and the numbers in it, its elements. 


Exercise 4 
Record the essential information about the following equations in matrix form. 


3x, —5x,= 8 Ey 
4x, + 7x, = 11 E, 
(Solution on p. 53) 


Example 5 

Solve the simultaneous equations 
2%, + X53 E, 
ax, +28, =35 oe 


This time, we shall use the matrix representing these equations throughout the 
elimination procedure. For brevity, we shall use R, for the first row of the matrix, 
and so on. 


The original equations can be represented by the matrix 
Z 1 3/R, 
eevee. 6 


Now E,, is found. by subtracting 3E, from E,. Arithmetically, this is the same 
operation as subtracting 3R, from R,. It is useful to record this operation in 
another matrix. The way in which the new rows are formed can be recorded on 
the left of the matrix: 


fs 1 
R, —5R, 0 : 


3]R, 
: R,, 


The important thing to note is that subtracting 3E, from E, produces the same 
coefficients as the numbers we store when subtracting 3R, from R,. To operate on 
the rows of the matrix in this way is less tedious than writing out the equations 
each time. 


Once the elimination has been done, the back substitution can be performed in the 
same way as before. Writing out the equations represented by the last matrix, we get 


2k, + Xy = E, (from R,) 
5X2 = 3 7 E,, (from R,,) 
so that, from E,,, 
Xx, =1. 
Substituting this into E,, we get 
2x, + 1 =3. 
Hence x, = 1, giving the solution 
x, =1, xX, =1. 
Checking this solution, 
LHS of E,: 2x, + x,=( x 1)+1=3= RHS. 
LHS of E,: 3x, + 2x, =(3 x 1)+- (2 x 1) =5 = RHS. 
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Exercise 5 
Use the matrix method to solve the simultaneous equations 
Xx, + 2x, =4 E, 
3x, — xX, =5 E, 


(Solution on p. 53) 


Example 6 


Solve the simultaneous equations of Example 4 using the matrix method. In 
matrix form, these give 


2 4 253 7) 8, 
8 6 | 13iR, 
= 3 


Stage 1. The elimination stage: 


Each part of the elimination stage in Example 4 has an equivalent part in matrix 
form. 


Stage 1(a) 
Z 4 —3 ‘16/R, 
R, —3R,|0 —8 4h —11/R,, 
R,;—3R,|0 —18 3 —66)R,, 
Stage 1(b) 
[2 4  -3 16/R, 
aS 
\ 21 
10 — a —11/R,, 
R,, i (S5R,, ee SS a R3, 


Note the triangle of zeros in the matrix which appears at the end of the 
elimination. This shows us that each equation has one less unknown than the 
previous one, which is what we need to be able to do the back substitution. 


Stage 2. The back substitution stage: 


This is exactly the same as Stage 2 of Example 4. 


Exercise 6 
Rewrite the solution of Exercise 3 in matrix form. 


(Solution on p. 53) 


Summary of Section 1 


In this section, we have seen a method for solving a set of linear simultaneous 
equations, called Gaussian elimination. 


The method has two stages: 


Stage 1. The elimination stage: this consists of systematically reducing the number 
of unknowns in each successive equation until we are left with a set of equations 
whose matrix representation has a triangle of zeros in the bottom left-hand corner. 
The method used is shown below. 


To obtain an equation E,, without a term in x, from the 
simultaneous equations 


a,X, + bx, +c,x, =d, 
a5, +O, +c, = @, 


(where the as, bs, cs and ds are given, and a, # 0), subtract 


2) _ from E,. The number “2 is called a multiplier. 


ay ay 
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Stage 2. The back substitution stage: this consists of finding the unknowns x,, x, 
and x; by solving the reduced set of equations which we obtained at the end of 
the elimination. These will have the form 


a,X, + b,x, +¢,x, =d, E, 
* * Shee ft 
bFX» + CiX3 = dj Ey, 
56 
c3x, = d3 E3, 


(i) Pind x, from £,,. 
(ii) Substitute this value of x, into E,,, and hence find x,. 
(iii) Substitute the values of x, and x, into E,, and hence find x,. 


The advantage of this method is that it is systematic, and can easily be extended 
to larger sets of simultaneous equations. The calculations can be simplified by 
using matrix notation. 


End of section exercises 


Solve the following sets of simultaneous equations using Gaussian elimination. Use 
a matrix to represent the equations for the elimination stage, and check your 
solution in each case. 


Exercise 7 

(i) x, —2x, =5 (ii) 2x, — 3x, = 16 
4x, + x,=2 3x, — 4x, = 22 

Exercise 8 

(i) x, -—3x,+ x,=11 (ii) 3x, — 2x, + 5x, = 10 
2X, + x,+4x,= 3 X, + 2x, + 3x3 = 10 
3x, — 5x, — 7x, = 11 x, + 5x, — 3x, = 10 


(Solutions on p. 53) 


2 Types of solution 


2.1 A geometric look at different types of solution 


(a) Linear equations in two unknowns 


The term ‘linear’ comes from the fact that the graphical representation of an 
equation of the form 


ax + by=c 
is a Straight line. 


For the purposes of this subsection, we shall refer to the unknown quantities as x, 
y and z (rather than x,, x, and x,), as these letters are traditionally used for three- 
dimensional Cartesian coordinates. 


When we write down two linear simultaneous equations in two unknowns 
a,x t+byy=c, Ey 
a,x + by =c, E, 


these can be represented graphically in two dimensions by two straight lines L, 
and L,. All the points on L, satisfy E,, and all the points on L, satisfy E,. Hence 
the intersection of the lines is a point satisfying both E, and E,. 
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The solution of the simultaneous equations can be thought of graphically as the 
coordinates of the point of intersection of the two lines L, and L,. If we draw two 
arbitrary lines, there are three situations which can arise, which are illustrated 
below. 


y 
(i) (ii) 
x 
L L, 
ax+by=c, a@x+by=c, L; L, o. 
ax+by=c, a,x +boy =c, ers, 


(1) The most likely situation to occur is that the lines will not be parallel, and 
will have one point in common. The coordinates of this point will also be the 
unique solution to the simultaneous equations. 


(ii) The situation when the lines are parallel occurs much less often. In this case, 
the lines don’t intersect at all, and the simultaneous equations will have no 
solution. 


(iii) The least likely situation to occur is that the two lines coincide. In this case, 
all the coordinates of the points lying on the line will satisfy the equation of the 
line, and the simultaneous equations will have an infinite number of solutions. 


So far, we have only met simultaneous equations which have a unique solution. 
We shall be looking at the kind of equations which have no solution, or which 
have an infinite number of solutions, in the tape subsection, Subsection 2.2. 


Exercise 1 

Sketch the graphs of the following pairs of simultaneous equations. 

as y= 4 (ji) x+ y=4 (ij) x+ y=4 
a 3y = 3 Le y= 5 2x+2y=8 


(The ‘solutions’ to these pairs of equations will be discussed in the tape subsection, 
Subsection 2.2.) 


(Solutions on p. 54) 


(b) Linear equations in three unknowns 
Again a linear equation in three unknowns 


ax +by+cz=d 


can be represented geometrically in three dimensions by a plane. If we consider 
three equations 


a,x+byt+ez=d, Ey 
a,x + boy + coz = d, E, 
a,x +b,y+c,z=d, E, 


these could also be thought of as the equations of three planes P,, P, and P3. 


By an argument similar to that of the two-dimensional case, the solution of three 
such simultaneous equations is the coordinates of the point of intersection of the 
three planes P,, P, and P,. Now, there are more ways of arranging three planes 
than there were of arranging two lines in two dimensions. Three of the possibilities 
are given over the page. 
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(i) A unique solution 


The planes P,, P, and P, can be seen to have only one point in common. In 
other words, the three equations in this case would provide us with a unique 
solution. 


(ii) No solution 


The planes P,, P, and P, clearly have no points in common. In this case the 
three equations would have no solution. 


(iii) An infinite number of solutions 


The figure opposite illustrates that the three planes could intersect in a 
common line. 


In this case the equations represented by the planes would have an infinite 
number of solutions, namely the points on the common line of intersection of 
the planes. 


Even if all the geometric possibilities are not exhausted here, they all correspond to 
one of the algebraic possibilities listed below. 


If we have n equations in n unknowns, then either 


(i) the equations will have a unique solution (which is the most frequent 
occurrence) or 


(ii) the equations will have no solution (which is rare) or 
(iii) the equations will have an infinite number of solutions (very rare). 


In the following Tape Subsection, we shall see how to determine the type of 
solution using the elimination procedure on any given set of n simultaneous 
equations in n unknowns. 


2.2 Finding the type of solution algebraically (Tape) 


Start the tape now. 


(ii) 


P, 
P; 


P 


(iii) 


Another geometric possibility 
is for all three planes to 
coincide. Can you think of 
any more? 
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SET ! 


Solve the simultaneous equations 


“2,+ Xo = 4 


2x, ~ 3x, 


is 


Stage |: the matrix after the elimination 
is 


Stage 2* the equations represented by 
this matrix ere 


and back substitution Gives 


from Eau 


from E, , 


Checking Solution in the original equations 
<, + SB = 


AG, 7 Sx, a 


15 


16 
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SET 2 


Solve the simultaneous Equations 


x,+ Z =4 E, 


2X, + RX. 2 


The matrix representing these equations 
is 


Stage |: the matrix after the elimination 
is 


Stage 2: the equations represented by 
this matrix are 


Equations 
tnconsistent 
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SET 3 


Solve the simultaneous equations 


= + oc, 


2x, + 2x, = 8 


The matrix representing these equations 
is 


Stage 2: the equations represented by 
Chis matrix are 


With only one equation 


K Ehen 


) 


Where K is an arbitrary number. 


Checking solution in original equations, 
xa ft xX, = 


2x, + ao = 
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SET 4 


Summary 


A set of simultaneous equations can have 


4) Q@ unique Solution 
4 * 
O / * 

(ii) NO Solution (When the equations 


are inconsistent) 


(ii) Qt infinite number of Solutions 
X22 


Key _ 


/ : must be non-zero 
#* =: can be anything 


MST204 9.2 Tape 19 


a6 TS 


Solve the sSimvuitaneous equations 


x,t ag cs 3 


AX, 


The matrix representing these equations 


Stage |: the etiminatior 


(a) 


Stage 2: the equations represented by this matrix are 


E, 
re 


Fab 


The back substitution: 


Ea, 


Checking the solution in 


3%, + 7x, + 2x 
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SET 6 


Solve the simultaneouS equations 
SS, ~ 220, + Sa, 


Ax, t3x, ~ “3 


x, tie 25205 + 24x 3 


Stage lt: the elimination 


(a) 


Stage 2: the equation E., represented by 


matrix iS 


These equations are inconsistent and Nave no solvtion. 
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SET 7 


Solve the simultaneous equations 
Sj es 3 
i + 2S + So, 


ys a -4a, 


The matrix representing these equations is 
Ry 
Ro 
a 

Stage|l: the elimination 

(a) 


Stage 2 : the equations represented by this matrix are 


The back substitution : 
let Xs 


from BE, Ky 


From E, x, 


Where kK is arbitrary - 


Checking this Solution in E3 


a2x,- X2-4x3 = 
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SET 8 


Summary _ 


The matrix formed at the end of the elimination will usually 
have one of the following characteristics * 


AN INFINITE NUMBER OF SOLUTIONS 


Key 


Y *: must be non-zero 
# * can be anything 


Should you need any more practice at determining types of solutions, try the 
following exercise. 


Exercise 2 

Determine the type of solution of each of the following sets of equations. Find the solution 
where possible. In the case of the set of equations with an infinite number of solutions, find 
the general solution. 


(i) 2X, —- X%,- xX,= 8 E, 
3x, — X%,-— 5x,= 6 E, 

X,+ %*,-11x,= 2 E. 

2X, +: Ate = AS E, 

x, + 14x, — 14x, = —27 i. 

(iii) 3X, hy RZ = £. 
2x, + 3x,-— 5x,= 4 £, 

ee ee rae eee a eee | Es 


(Solutions on p. 54) 
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2.3. Linear dependence 


Before discussing this topic, it is convenient to introduce some new notation. 
Suppose we have a set of equations 


@,x%, + 6,x, + ¢€,x, =d, Es 
a,x, + b,x, + ¢,x, =d, EE, 
GX, + 64%, +.€4X, = d; Es 


Sometimes we want to refer to the matrix of the left-hand side coefficients, 
sometimes to the matrix of the numbers on the right-hand side of the equations, 
and sometimes to the whole matrix. In future, we will refer to the matrix of the 
left-hand side coefficients as A; that is, 


ay b, Cy 
a3 b; C3 


Similarly, we will refer to the matrix of the right-hand sides of the equations as b; 
that is i‘ 

b=, F 

d; 

Finally, we shall refer to the matrix which contains both sets of information as 
A|b; that is, a, b, c, d, 

A|b = | a, b, "ae d,|. 

a; b, C; d, 

This of course is the matrix which we use to do the elimination. 


We saw in Subsection 2.2 that it is possible to tell what type of solution a set of 
simultaneous equations will have, by inspection of the matrix obtained after the 
elimination. The following examples examine the elimination procedure in more 
detail to see how a row of zeros might occur. 


Example 1 
If you have to solve any set of simultaneous equations—for example 
X,+ x,-— x,=3 E, 
2h hs =  X, = 0 is 
5X, 1k, + 2%, = 5 KE 
the first thing you do is to write down the matrix A|b: 
1 1 —1] 3/R, 
2 —3 —1] 0O;R, 
3 7 2 8|R, 


Then you go through (in the case of three equations in three unknowns) the two- 
stage elimination procedure: 


Stage 1(a) 
1 1 —| 3|R, 
R, — 2R,|0 —5 1 —6/K.., 
R, — 3R, [0 4 5 —1|R,, 
Stage 1(b) 
1 1 —] 3 |R, 
0 —5 1 —6 R,, 
R,, — (—3)R,, 0 0 2 aes R3, 


This is a matrix with 
one column. 
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At Stage 1(b), inspection of the matrix tells you that you are going to have a 
unique solution, as there is no problem with the back substitution. 


Let us look more carefully at how we obtained R,;,. We know from Stage 1(b) 
that 


R3, = R3,. + $Ro,. (1) 
But we also know from Stage 1(a) that 

R,, = R, — 2R,, 

R,, = R, — 3R,. 
If we substitute these expressions for R,, and R,, into (1), we get 

R3, = R,; — 3R, + 3(R, — 2R,) 

= —3R, + GR, + R3. 

You can see that, independent of the particular elimination we have done, R3, is 
always given by an expression of the form 

R,, =k,R, + k,R, + KR, 
(where the ks are numbers—not all zero). 


So R3;, is a linear combination of the rows R,, R, and R, of the original matrix 
A|b. You have met linear combinations before, in Unit 6. In fact, each new row we 
form during the elimination process is made up of a linear compaion of the 
rows of A|b. For instance, in the last example, 


Sometimes, one of these linear combinations may result in a row of zeros, as the 
following example will demonstrate. 


Example 2 
If we go through the same procedure as in the last example with the equations 
X,+ X,+ xX3,= 3 Ey 
X, + 2x,+3x,= 6 oe 
2x, — x, —4x%, = —3 E, 
the elimination goes as follows: 
Stage I(a) 
1 1 1 3/R, 
R, —R, |0 1 2 3/R,, 
R, — 2R,|0 —3 —6 —9IR,, 
Stage 1(b) 
1 1 1 3|R, 
0 1 2 3iR,, 
R;,—(—3)R,,10 0 0 | OJRs, 


The row of zeros tells us that this set of equations will give us an infinite number 
of solutions. 


This time, we know that 


R;, ea R;, 5 3R,, 


where 

R,, = R.-Ri 
and 

R= BR; 2h, 
giving 


R,, = —5R, + 3R, + R;. 


The equals sign in this context 
means ‘is made up of’. 
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So we know that for this set of equations, 
—5R,+3R,+R;=0 (a row of zeros). 


It can be shown that for any equations which have an infinite number of solutions, 
there will be some linear combination of the rows of A|b which is equal to a row 
of zeros: 


k,R, + k,R, + k,R,; =0 


(where the ks are numbers, not all zero). 


When such a relationship exists between the rows of a matrix, we say that the 
rows are linearly dependent. 


Rows R,,R>,...,R, of a matrix are said to be linearly dependent if 
some linear combination is equal to a zero row: 


k,R,+k,.R,+°°°-+k,R,=0 


where the ks are numbers, not all zero. 


If the rows are not linearly dependent, we say that they are linearly independent. 
Again, you have already met the concept of linear independence in Unit 6. 


To complete the picture, let us have a look at the case of a set of simultaneous 
equations which have no:solution. 


Example 3 
If we do the elimination procedure for the equations 
Me 5X, ee E; 
2 ks hg ee E, 
X, — 23x, + 24x, = 18 E, 
we get 
Stage 1(a) 
1 —2 3 O;R, 
R, — 2R,|0 7 —7 4'R,, 
R, —R, [0 —21 21 ink: 
Stage 1(b) 
1 —2 3 O|R, 
| 0 7 —7 4\R,, 
R,,—(75R,,10 0 0 | 30)Rs, 


It is the row (0,0,0,30) which tells us that these equations are inconsistent and 
have no solution. Now, this time (as for most other sets of inconsistent equations), 
we can see that 


(i) the rows of A are linearly dependent but 
(11) the rows of A|b are linearly independent. 


A certain amount of common sense has to be exercised when interpreting this type 
of solution. For instance, the equations 


oe tea eg eZ i 
2k; + 2X, + 2x, = 4 E, 


give us the following matrix after elimination: 
1 1 1 1|R, 
R, — 2R,| 0 0 0 0|R,, 
R; — 2R,|0 0 ) 2/R;, 
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This is a rare case where you can see both the row of zeros that you would expect 
to see for an infinite number of solutions (R,,), and also the situation that you 
expect to find for inconsistency (R;,). Faced with both these possibilities, the 
inconsistency must take priority, as at no time in the process of solving these 
equations is it possible to obtain an equation 


Ox, + Ox, + 0x, =2 


without the equations being inconsistent. We can express this in terms of the ideas 
of this subsection as follows: if a certain linear combination of the rows of A is 
equal to zero, and the same linear combination of the rows of A|b is non-zero, 
then the equations are inconsistent. On the other hand, if this situation is not 
present, then any linear dependency guarantees an infinite number of solutions. 


We shall use these ideas of linear dependence in the next section, and again in the 
units on matrices and eigenvalues which you will meet in the course. 


Exercise 3 
The equations which you solved in Exercise 2(ii), 
X,—-— 2x,+ 4%,= 7 rE. 
2x, + 4x,-— x,=-— 3 E, 
x, + 14x, — 14x, = —27 E, 


have an infinite number of solutions. Use the elimination procedure to determine what 
linear combination of the rows of A|b gives a row of zeros. 


(Solution on p. 55) 


The ideas in the last three examples can be summarized as follows. 


Given a set of equations represented by the matrix A|b, as 
explained on p. 23, then: 


(i) if the rows of A are linearly independent, then the equations 
have a unique solution; 


(ii) if the rows of A are linearly dependent, and the rows of A|b 
are linearly independent, then the equations are inconsistent, and 
have no solution; 


(iii) if the rows of Alb are linearly dependent, then the equations 
usually have an infinite number of solutions. 


Summary of Section 2 

1. Given a set of linear simultaneous equations, they will have either: 
(i) a unique solution (the most common situation), or 

(ii) no solution (rare), or 

(iii) an infinite number of solutions (very rare). 


2. The rows R,,R,...R, of a matrix are said to be linearly dependent, if some 
linear combination of these rows is zero, 


ie. constants k,,k,,...,k, (not all zero) exist such that 


k,R, +k,R,+-::+4,R, = 0 (row of zeros). 
3. To find out which type of solution a particular set of simultaneous equations 
has, do the elimination automatically on the matrix A|b. The matrix formed at the 
end of the elimination will usually have one of the following characteristics. 
[In the following table, a matrix entry which must be non-zero is denoted by a tick 
(,/), while a matrix entry which could be zero or non-zero is denoted by a star 


(*). ] 
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Characteristics of matrix 
obtained after elimination Type of solution 


of 7 * ey unique solution 


= x * * | no solution 


| (equations 
" J | inconsistent) 


a zero row of A with 
the RHS entry non-zero 


2 ** * * an infinite 
number 
0 J . . : of solutions 


Seem merece eres ee sees ese ee eeeeeeeeeeeeteeeesy 


Reece eee ee cress eee eeeeseseseseesseseetensesll 


a zero row of A\b 


End of section exercises 
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Comments 


The rows of A are 
linearly independent. 


The rows of A are linearly 
dependent, whereas the row 
of Alb are not. | 


The rows of Alb (and 
hence also of A) are 
linearly dependent. 


In the following exercises, determine the nature of the solutions of the following 
sets of equations. In the case where the equations have a unique solution, find it. 
In the case where the equations have an infinite number of solutions: 


(i) find the general solution; 


(ii) find which linear combination of the rows of Alb gives a row of zeros. 


Exercise 4 
ho 2k, + Oe, 
X,+ 3x,-— 4x, =20 
x, + 18x, — 31x, = 40 


Exercise 5 
X, = 2x, + 3x, = 6 


Exercise 6 
9hj = he A Se 
x; = &, + ayes 
6x, + 14x, — 6x, = —52 


(Solutions on p. 55) 
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3 Some possible difficulties (Television Section) 


3.1 Preparation for the television programme 


In this unit we have used the systematic method of Gaussian elimination to solve 
sets of simultaneous equations and, to date, nothing unpleasant has happened. 
However, this is not always the case, as you will see in the television programme 
and later in this section. 


The first part of the television programme (and Subsection 3.2) discusses sets of 
equations which are ill-conditioned. The equations in the first part of the following 
exercise relate to the ‘practical’ problem discussed at the beginning of the 
programme. The second part of the exercise is strictly speaking post-television 
work, but we thought you would be interested to see these two problems together. 


Exercise 1 
(i) Verify that the solution of the set of equations 
6b+ 4¢g+4+ 31= 735 E, 
746+ 6¢+ 51 = 1052 E, 
10b + 73g + 61 = 1331 E3 


b= 74, ¢< —18, f= 121. 


(ii) Verify that the solution of the set of equations 


6b+ 4¢+31= 734 oe 
7T4b + 6g 4+ 51 = 10495 . 
10b + 74g + 61 = 1331 Ee: 


b = 53, ¢ = 62, 1 = 56. 


(Note that the equations in (ii) are similar to those in (i), except for small changes to the 
right-hand sides of E, and E,.) 


(No solution given.) 


The second part of the television programme discusses the effect of rounding error 
in sets of equations which are not ill-conditioned. You may have noticed that so 
far in this unit, fractions rather than decimals have been used to solve the 
problems. This has been done to avoid introducing any error into our calculations, 
and each time we have obtained the exact solutions of the sets of equations. This 
is not the usual state of affairs. Normally we use a calculator or computer to help 
us do these problems. These machines hold approximations of numbers—albeit 
that they are very accurate approximations. The result is that the solution that we 
obtain using a computer is not exact. 


The second problem in the programme is done in two ways. The first time it is 
done using exact arithmetic (i.e. fractions}—and the second time ‘three figure 
arithmetic’ is used. This means that each number at each stage of a calculation is 
rounded to three significant figures. The following example demonstrates how this 
arithmetic works. 


Example 1 
Suppose that we wish to evaluate 
(34.87 x 0.018 54) — 8.575 


using three figure arithmetic. First, each number is rounded to three significant 
figures. 34.87 is rounded to 34.9, 0.018 54 is rounded to 0.0185, and 8.575 is 
rounded up to 8.58. 


Iv? 


You met ill-conditioning in a 
different context in Unit 1. 
The exact definition for the 
present context is given on 
page 29. 


In the television programme 
round rather than square 
brackets are used for 
matrices. In fact both types of 
notation are in common use. 
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So, 34.87 x 0.018 54 is calculated as 

34.9 x 0.0185 

= 0.645 65. | 
This figure is rounded to 0.646 before the calculation is completed. So finally, 


(34.87 x 0.018 54) — 8.575 
~ (34.9 x 0.0185) — 8.58 = 0.645 65 — 8.58 
~ 0.646 — 8.58 = —7.934 
~ —7.93 (to three significant figures). 


Comparing this with the exact answer of —7.9285102, you can see that the result 
is still quite accurate, considering that only three significant figures were retained 
at each stage of the calculation. 


Exercise 2 

Use three figure arithmetic to evaluate 
(0.2745 x 73.841) — 6.1879. 

(Solution on p. 55) 


Again, in this exercise, three figure arithmetic gives us a good approximation to 
the answer, but this is not always the case, as you will see in the television 
programme. 


Although a computer or calculator works far more accurately than this, the 
arithmetic used in these machines is analogous to the process described above. So 
the difficulties which arise using three figure arithmetic will also arise when using 
computers or calculators. 


Now watch the television programme. Concentrate on the methods of solution which 
are shown, rather than on the arithmetical details. The following subsections all 
contain reference to the television programme. Specific references will be marked with 


a \( tv): symbol. 
3.2 Ill-conditioning 


The definition of ill-conditioning of a set of linear simultaneous equations is similar 
to that given in the unit on recurrence relations. 


A set of linear simultaneous equations is said to be ill-conditioned 


if small changes in the data lead to large changes in the solution. 


The ‘data’ in the case of simultaneous equations refers either to the coefficients of 
the left-hand sides or to the right-hand sides of the equations. 


Normally, if slight changes are made to the data in a set of simultaneous 
equations, then the solution only changes slightly, as the following example 
demonstrates. 


Example 2 

You can check that the simultaneous equations 
x+y= 5 
x—2 = —1 


have the exact solution 
x = 3, yen 2, 


Suppose, now, that slight changes are made to the right-hand sides of these 
equations, and we solve the equations 


xo ye 5A 
x —2y = -—1.01 
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These equations have the exact solution 
eee 01, y= 2.01, 


which as you can see is very close to the original solution. The geometric 
interpretation of small changes made to the right-hand sides of these equations is 
shown in the figure below. 


De ae A) 


The figure above shows the two lines. The dotted lines indicate how much the 
lines might vary if there were small changes in the right-hand sides of the 
equations, and the shaded area indicates the region in which we would expect the 
solution of our equations to lie if these small changes were made. As you can see 
in this case, the solution we would get to an inexact problem would be a 
reasonable approximation to the solution of the accurate problem, as all the 
points in the shaded region are reasonably close to x = 3, y = 2— the exact 
solution. 


Usually, when small changes are made in the data, the solution of the resulting 
problem is very close to that of the original problem. When solving problems we 
depend on this fact as in any practical context there would almost certainly be 
small errors in the coefficients of sets of equations, and yet we have to rely on the 
solution of this inexact problem being close to the accurate one. 


However, this is not always the case, as the following example will demonstrate. 


Example 3 
You can eheck that the simultaneous equations 
x— 2y=-1 E, 
10x — 19y = —8 E> 


have the exact solution. 
x = 3, yey, 


Once again, we shall make small changes to the right-hand sides of the equations. 
Solving the slightly changed equations 
x- 2y=-I1.1 E, 
10x — 19y = -—8 E, 
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we find that the exact solution to this slightly altered set is 
x = 4.9, ee ee 


This time, the solution of the slightly altered set of equations bears no resemblance 
to the solution of the original equations, and we see that the equations are ill- 
conditioned. The geometric interpretation of this situation can be seen in the figure 
below. 


6.9 


As the equations are represented geometrically by two nearly parallel lines, 

we have not drawn in all the lines corresponding to those in the figure for 
Example 2, because this would make the diagram confusing. This figure simply 
illustrates the remarkably elongated region in which you could expect the solution 
to lie if small inaccuracies were present in the right-hand sides of these particular 
equations. This shows that the solution of these slightly inaccurate equations could 
in fact be a long way from the accurate solution x = 3, y = 2. In other words, 
with this particular pair of equations, a small variation in the data could produce 
a large change in the solution. 


It is obviously very important to know whether a set of simultaneous equations is 
ill-conditioned. If it is, the solution obtained should not be considered as reliable. 


The geometry shows us that if the lines representing two simultaneous equations 
are nearly parallel, then the equations will be ill-conditioned. Whereas this 
concept cannot really be extended to larger sets of equations, the equivalent 
algebraic concept can. The gradients of the two lines are obtained from the left- 
hand side coefficients of the equations. If we write these coefficients in matrix form 


1 —2 JR, 
a= 
10 —19 JR, 
then an equivalent algebraic way of saying that the lines are nearly parallel would 


be to note that R, is nearly a multiple of R,. (R, is roughly nine or ten times as 
big as R,.) In other words, the rows of A are nearly linearly dependent. 


In general, if a set of linear simultaneous equations is represented by a matrix Alb, 
then if the rows of A are ‘nearly’ linearly dependent, ill-conditioning is likely to 
occur. 


As you saw in Section 2, linear dependence will show up as a row of zeros if 
Gaussian elimination is performed on a matrix. So if the rows were nearly linearly 
dependent we would expect @ row in the elimination to be ‘nearly’ zero. 
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Perhaps we should mention at this stage that, as in the case of recurrence relations 
in Unit 1, there are two kinds of ill-conditioning. 


Suppose that a change is made in the data of a set of simultaneous equations. 
Then these equations are: 


(i) relatively ill-conditioned if the relative change in the solution is much larger 
than the relative change in the data; 


(ii) absolutely ill-conditioned if the absolute change in the solution is much larger 
than the absolute change in the data. (In the case of simultaneous equations, it is 
usually relative ill-conditioning which concerns us.) 


Now, ill-conditioning would not be a problem if we could be certain that the data 
in the equations were exact. In this unlikely event, we could do exact arithmetic to 
get the exact solution. It is rather more usual for the coefficients to have come 
from some other mathematical process, or be the results of some physical 
experiment. In the case of the mathematical process, rounding error would already 
be present. In the case of a physical experiment, the data would certainly contain 
experimental error. 


Example 4 
The equations 


1.0x, + 2.0x, = 3.0 
2.0x, + 4.02x, = 6.02 


have the exact solution 
X14 = i. X72 = ¥ 


If the data in these equations had been the result of some experiment, then the 
reading 4.02 might well have been read as 3.98—an error of only 1%. So we could 
just as easily have landed up with the equations 


POx. + - 20x, = 30 
2.0x,; + 3.98x> = 6.02 


which have the exact solution 
4S 5, ST ate z 
Quite a different solution! 


It should be noted that the large change in the solution cannot be attributed either 
to the method used to solve the problem, or to the arithmetic, as the answers 
given are the exact solutions of the problems. The general problem of detecting ill- 
conditioning is difficult. Sometimes, as in the bar problem on television, you get 
results which are clearly impossible. The price of Guinness couldn’t possibly be 
—18p and thus you are alerted to the fact that something is wrong. However, that 
was lucky! The direct approach to detecting ill-conditioning is to vary the data 
slightly, and see if the solution of the altered equations is very different from the 
original one. This would be very tedious without the use of a computer. You will 
be given the opportunity to do this using the computer package in the next 
section. 


The cure for ill-conditioning is fairly drastic. We can abandon the equations in 
hand, and try to find some more data—or even abandon the whole model. An 
attempt to find new data was made in the bar problem by ordering another couple 
of rounds of drinks ... 


3.3. The problem of rounding errors 


As you saw in Subsection 3.1, a calculator or computer introduces error into a 
calculation. To start with, these machines have to work with approximations to 


numbers like 4 or is and then as the calculation (Gaussian elimination, for 
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example) progresses, at each step rounding takes place. After a while, most of the 
numbers in the calculation will have rounding error in them. It becomes a matter 
of prime concern to try and prevent this error (and any other error present) from 
building up to the point that the answers obtained bear little or no resemblance to 
the correct solutions. 


In Subsection 3.1, the simple example and exercise using three figure arithmetic 
produced an answer which was a good approximation to the exact answer. 
Unfortunately, there are circumstances in which this is not the case. 


Example 5 
(,/2)? — 1.98 


It can be seen that the exact value of is 2, whereas using three-figure 


0.01 
arithmetic, 
(/2)-198 (1.41)? — 1.98 
0.01 - 0.01 
1.99 — 1.98 
7 as 
= 1. 


Thus a very small error made in measuring one (0.3%) has produced a 50% 
change in the answer. 


The story of error and error build-up is an extremely complicated one. In this 
subsection, we look at just one particularly bad situation which can arise when 
solving simultaneous equations. 


Example 6 
The television programme discusses the solution, using three figure arithmetic, of 


the problem 
x — ,/2y + ee | 
—./2x + 2y =i 
2y — ./2z = 1 


Using exact arithmetic, this produces a solution 


2 


— 


= 1.71 (to three significant figures). 
The only rounding done in this process is in the final form of the answer. 


Now, using three-figure arithmetic to do this same problem, the matrix 
representing these equations is 
1.00 —1.41 1.00 1.00/R, 
—1.41 2.00 0 1.00/R, 
0 2.00 —1.41 1.00} R, 


The elimination goes as follows: 


Stage 1(a) 
1.00 —1.41 1.00 1.00 /R, 
—141 |] 0 0.01 1.41 2.41 |R>, 
0 0 2.00 —1.41 1.00 | R;, 
Stage 1(b) 


100 —1.41 1.00 1.00} R, 
0 0.01 1.41 2.41/R>, 
200 0 0 —283 |—481 | R3, 
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Note that just a record of the multipliers has been kept at each stage of the 
elimination. For example, in Stage 1(a), where before we would have written the 
comment ‘R, + 1.41R,’ on the left of the matrix opposite R,,, we now just record 
the number — 1.41, being that multiple of R, which is subtracted from R, to 
produce R,,. We shall stick to this notation for the rest of this unit. 


Doing the back substitution: 
R3, gives 

— 283z = — 481, 
hence 

z = 1.70 (not bad). 


R,, gives 
0.0ly + 1.412 = 2.41, 
hence 
2.41 — (1.41 x 1.70) 
a 0.01 
= 1.00 (very bad); 


and finally R, gives 


1.00x — 1.41y + 1.00z = 1.00 
hence 
x = 1.00 + (1.41 x 1.00) — (1.00 x 1.70) 
= (0.710 (hopeless). 


The solution 

x = 0.710, y == 00. z= 170 
is nowhere near the solution of the exact problem. As this is a disastrous state of 
affairs, it is worth analysing in more detail. 


To help understand what went wrong in this particular problem, let us first look 
at the graphical interpretation of Stages 1(a) and 1(b) of the elimination process, 
for a more typical case in which nothing goes wrong. 


Stage l(a) Stage 1(b) 


When Stage 1(b) of the elimination is done, the line representing the equation E3, 
is replaced by a line parallel to the y-axis. The lines representing E,, and E3, have 
the same point of intersection as those representing E,, and E3, (see the diagram 
above). If the arithmetic were exact the solution of these two pairs of equations 
would be exactly the same. 
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Now let us look at what happened in Example 6. 


Stage 1(a) Stage 1(b) 


In Stage 1(a) in Example 6, we had the equations 


0.0ly + 141z =2.41 i. 
2.00y — 1.41z = 1.00 ae 


From the diagram above, it can be seen that the trouble arises because E,, is 
practically parallel to the y-axis. Thus, although the equations in Stage 1(a) are 
not ill-conditioned, the ones we obtain in Stage 1(b) are, as E3, is always parallel 
to the y-axis. Now, inaccuracy has been brought into the calculation from: 


(i) approximation of 2 as 1.41, 
(11) the use of three-figure arithmetic; 


so we would hardly expect the resulting solution to bear any resemblance to the 
solution of the accurate problem. 


Whereas ill-conditioning was a property of the equations we started with, this 
situation is rather different. Here, the ill-conditioning is induced by the method we 
used to solve the equations. We began with a problem which was not ill- 
conditioned to start with, and yet the method induced ill-conditioning. This 
induced ill-conditioning is often referred to as induced instability. 


In order to cope with larger sets of equations, we need to look for an algebraic as 
well as a geometric explanation. 


One way of seeing what went wrong between Stage 1(a) and Stage 1(b) is to look 
at how R3;, is derived. 


100 —141 — 100| 1.001R, 
0 Oot 441 2ai ie, 
R208, [0 3 | at 8. 


We know that 
R3, = R;, — 200R,,. 


Now 200R,, is likely to swamp any information held in R,, because of the large 
size of the multiplier, so 


R;, ~ —200R,,. 


The large multiplier has ensured that R,, and R3, are nearly linearly dependent, 
and hence are likely to be ill-conditioned. 
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The argument above suggests that if we want to avoid inducing instability, we 
must try to adapt the method of Gaussian elimination to avoid large multipliers. 
The way in which this is usually done was briefly demonstrated in the television 
programme, and is called partial pivoting. This will be discussed in more detail in 
the next subsection. 


3.4 The strategy of row interchange 


It would be convenient in this subsection to have some extra terminology. At each 
stage of the elimination, we have to divide by a number to form the multipliers. 
The numbers we divide by are often referred to as pivots (or pivot elements), and 
the rows in which they lie as pivot rows. 


Example 7 


Given the set of equations represented by the matrix 


t 41738. 
o> 4 Se 
5. 2] 2 


then 1 is the first pivot, and R, the first pivot row. After the first stage of the 
elimination is done, the second pivot, 2 in this case, lies in the second pivot row 
R,,. 

1 1 1 3 |R, 


O12 34-518, 


6 =4 351-5 2. 


The subsequent pivots can be found after further stages in the elimination. 


(a) Essential row interchange 


In Subsection 3.3, we saw that it was advisable to avoid large multipliers. 

Now, as large multipliers are caused by small pivots, we shall have to avoid small 
pivots. To start with, let us look at what happens when we meet a zero pivot. 
When we meet a zero pivot, the idea of altering the order of the equations (or the 
equivalent rows in a matrix) is fairly instinctive, as the following example 
demonstrates. 


Example 8 
Given the set of equations 
Xp +x3=2 E, 
2x, + 4x, — x3 =5 E, 
X, + 2x, —x3 =2 E3 


represented by the matrix 


ee 
°° 4 3 
i: 3S ee. 


it would be impossible to do Gaussian elimination on the matrix as it stands. The 
reason for this is that the first multiplier would be 0° which is of no use. These 


equations have a solution x, = 1, x,=1, x3 =1, and, as you will see later, this 
is a unique solution. So the elimination process as described so far would not cope 
with this situation. 


The trouble arose because there was a zero pivot. In other words, the equation 
X, + x3 = 2 is not at all suitable to have as the first equation. The obvious remedy 
is to change the order of the equations. This is equivalent to interchanging two 
rows of the matrix Alb. If we were doing the problem above, we could interchange 
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two rows to obtain the matrix 


24 i bs 
ae ee. as 
eo ee 


We could then proceed with the elimination to obtain 


t 4) 1 Se 
ie £417 Se 
3,9 O —4 |-4]Rs, 


and hence obtain a unique solution. 


Where a row interchange has taken place, a double-headed arrow will be used to 
demonstrate which rows have been interchanged, but the rows will keep their 
sequential numbering. 


To be able to use Gaussian elimination on a set of equations, we must check at 
each stage that the appropriate pivot is not zero. If a zero pivot is encountered, an 
essential row interchange has to be made, such as the one we have just seen. 


The instruction of which row to interchange with which has got to be general 
enough to cover all possibilities. For instance, exchanging R, and R, would not 
work in the case of the equations 


Xp XZ 
X, + 2X3 53 
—2x; — xX, + 5x3; =2 


as once again, we would get a zero pivot. 


The general rule we adopt for essential row interchange is as follows: 


If any pivot element is zero, interchange this pivot row with the 
row containing the coefficient of highest modulus in the column 


underneath the pivot. 


Then, short of the very unlikely possibility of the whole column being zero, this 
will produce a non-zero pivot. 


Note that this process of essential row interchange can be done at any stage of the 
elimination, as you can see in the next exercise. 


Exercise 3 
Use Gaussian elimination with essential row interchanges to solve the equations 


X1— Xg+ Xz3— Xye=4 E, 
2X, — 2x2 + 3x3 4+ 2x4 = 5 E, 
X, — 2x3 — 3x, =0 E; 
—X,+4x3;+ x,=4 E, 


(Solution on p. 55) 


Example 9 
To investigate the type of solution of the equations 
xy + Xe + Xn = 3 
Xy + X2+2x3=4 
Xi Xe + 3X = OG 
for various values of g, once again we use the elimination process. 


Stage 1(a) gives us 
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In a rare event such as this, it is impossible to find a non-zero pivot, and the 
elimination process as described has broken down. At this stage, R2, gives us 
Xa = 1. 
On the other hand, R3, gives us 
2x3 =q — 3. 


So, if these equations are consistent, then q = 5. If q has any other value, then the 
equations will be inconsistent. The reason that the possibility of inconsistency did 
not show up in the elimination was that we were not able to complete the process. 


Finally, it is instructive to look at what would have happened had exact arithmetic 
been performed on the second television example (i.e. Example 6 on p. 33). The 
matrix representing the equations is 


1 —./2 ee 
2 2 O.| 1) ® 
0 ee fe es 


The first stage of the elimination would give us 


ee 1 R, 
2) Oe J/2 | tee ee 
0 bo 2 R3, 


In this case, we would now have to make an essential row interchange to be able 
to continue the process. The equivalent stage in the inexact arithmetic was 


1 —1.41 1 1 R, 
-141 | 0 (001) 141] 2.41] R.. 
0 0 F  -~141 167m. 
With hindsight, perhaps we should have realized that this small pivot element was 
very dangerous, and we should have changed the rows over anyway—even though 
the pivot wasn’t zero. Certainly the existence of this small pivot (0.01) caused the 


large multiplier (200)—which in turn induced instability into a problem where 
there was none to start with. 


(b) Partial pivoting 


z 


Stage 1(a) Stage 1(b) 
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In the second television example (Example 6), the first stage of the elimination 
gives us the equations 


0.0ly + 1.41z = 2.41 E>, 
2.00y — 1.41z = 1.00 E3, 

Had we interchanged the order of these equations (see figure on p. 38) 
2.00y — 1.41z = 1.00 E,, (new) 
0.0ly + 1.41z = 2.41 E3, (new) 


and performed the second stage of the elimination on them in this new order, then 
the problem of ill-conditioning would not have arisen, as you can see from the 
diagram. Thus, the solution we would get using the equations in this new order 
would be reasonably accurate. In an attempt to avoid inducing ill-conditioning 
into a problem we employ a strategy called partial pivoting. This is the same as the 
procedure described for essential row interchange, except that we interchange the 
rows (if necessary) automatically at each stage of the elimination. As before, at the 
beginning of each stage of the elimination, we choose to interchange two rows of 
the matrix, so that the pivot has the highest possible modulus. This should ensure 
that the multipliers are all less than or equal to one in modulus. 


Partial pivoting is the process of automatically altering the order 
of the equations at the beginning of each stage of the elimination 
in the following way. 


Interchange the current pivot row with the row containing the 
coefficient of highest modulus in the column underneath the pivot. 


Example 10 
We shall solve the equations from the television programme again: 
1.00x — 1.41y + 1.00z = 1.00 E, 
—1.41x + 2.00y = 1.00 E, 
2.00y — 1.41z = 1.00 E, 


This time, however, we shall use partial pivoting as well as three-figure arithmetic. 
The matrix representing the equations is 


1.00 —1.41 1.00 1.00 R, 
—1.41 2.00 0 1.00 R, 
0 200° 114i 1.00 R; 


Before performing the first stage of the elimination, interchange R, and R,, as 
— 1.41 in R, is the coefficient with the highest modulus in the first column. This 
gives us 


—1.41 2.00 0 1.00 R, eS 
1.00 —1.41 1.00 1.00 R, 
0 2.00 —1.41 1.00 R, 
and then we do the first stage of the elimination: 
—1.41 2.00 0 1.00 R, 
— 0.709 0 0.01 1.00 £7) R>, 
0 0 2.00 —1.41 1.00 R;, 


Before the second stage of the elimination is done, we interchange R,, and R;, to 
obtain the coefficient with the largest modulus for the second pivot. This gives us 


~1s 300 6 1.00 ] R, 
0 200 —14)- 9 40e Re 
0 et 160 § 171 | Re 
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and hence the second stage of the elimination: 


1.41 2.00 0 1.00 R, 
0 2.00 —1.41 1.00 R>, 
0.005 0 0 1.01 L7t R3, 
The back substitution gives us 
L0iz = Lat. 


Hence 


z = 1.69 (to three significant figures). 


From R,,, we get 
2y — 1.41z = 1.00. 
Hence 


_ 100+ (141 x 1.69) 
= 2 
= 1.69 (to three significant figures). 


Finally, R, gives us 


—141x+ 2y =1. 
Hence 
= 1 — (2 x 1.69) 
ee 


= 1.69 (to three significant figures). 


This solution x = y = z = 1.69 is a far better estimate than that obtained without 
partial pivoting. 


Finally, note that partial pivoting is not an absolute safeguard against induced 
instability. It does have the advantage of being easy to implement and is a good 
‘rule of thumb’. If, in addition, we try to ensure that the coefficients of the 
equations we start with are roughly of the same size, then partial pivoting usually 
works. This can often be done by multiplying the original equations by a suitable 
factor. This is called scaling the equations. Equations which arise in practice are 
very often naturally scaled. 


The process of partial pivoting used with Gaussian elimination 
ensures that the largest possible value is used as a pivot at each 


stage of the elimination. This usually helps to control the build-up 
of error during the process. 


Summary of Section 3 

1. The section introduces some new terminology. 

(i) The kth pivot (or pivot element) is the one by which we divide to obtain the 
multiplier in the kth step of the elimination. 

(ii) The pivot row is the row in which the current pivot lies. 


(iii) Essential row interchange is the process of interchanging the rows of a 

matrix when the potential pivot is zero. This can be done by interchanging the 
current pivot row with whichever of the rows beneath it will provide the pivot with 
the largest modulus. 

(iv) Partial pivoting is the process described above, with the difference that this 

is done at each stage of the elimination. 


2. The section shows some of the difficulties presented by simultaneous equations 
and their solution. 
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(i) Ill-conditioning. A set of equations is said to be ill-conditioned if a ‘small’ 
change in the data (the coefficients, or the right-hand sides of the equations) 
produces ‘large’ changes in the solution. Ill-conditioning is a property of the 
equations themselves. One way to detect the condition is to vary the coefficients 
slightly and see if there is much change in the solution. A possible cure for the 
situation is to obtain some more independent information, and hence obtain a set 
of equations which are not ill-conditioned. 


(ii) Induced instability. This is a difficulty where we start with a set of equations 
which are not ill-conditioned and yet, due to the method used to solve the 
equations, ill-conditioning is induced into the problem during the process of 
solution. To try to avoid this happening (and to avoid the subsequent build-up of 
error), Gaussian elimination with partial pivoting is used. 


3. The following flow charts might serve as a useful guide as to how to proceed 
when systematically solving a set of n linear simultaneous equations in n 
unknowns. i 


(1) The elimination process for n equations in n unknowns 


Start with the top left-hand element 
of Alb as the first potential pivot 


Employ partial pivoting to ensure 
that the element of largest modulus 


in the present column is used 
as a pivot 


Is The equations 


the pivot as have no solution, 
zero? or an infinite 
number of solutions 
No 
Test the equations 
Is to see if they are 
the pivot Yes ill-conditioned 
‘near’ zero? : 
No 
| Is Yes On to back 
this the substitution and 
last row? unique solution 
No 


Subtract a suitable multiple of the 
current pivot row from each row below 


to obtain a column of zeros below 
the pivot 


For next potential pivot, move down 
a row and one column to the right 
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(ii) The back substitution process for n equations in 7 unknowns 


After the elimination is finished, 
start with the last equation 


Substitute any ‘unknowns’ whose 
values have been found so far 
into this equation 


Solve for the remaining ‘unknown’ 
in this equation 


Write out 
the solution 
to the equations 


No 


Go back one equation 


End of section exercise 


The exercises on ill-conditioning are included in the next section, where you can 
use the computer package to help you. 


If you have time, do the following exercise now. If not, use the computer package 
to solve the problem on your next terminal visit. 


Exercise 4 

Check that the equations 
1.62x, + 1.10x, + 0.65x3 = 3.37 E, 
6.18x, + 4.20x, — 3.04x3 = 7.34 E, 
4.65x, — 3.05x2 + 2.10x3 = 3.70 E, 


have the exact solution 
xy = Xa = X54 eS 
Use Gaussian elimination to solve these equations: 
(i) without partial pivoting, 
(ii) with partial pivoting. 
Do each stage of the arithmetic to four significant figures. 


(Solution on p. 56) 


4 The computer package 


4.1 Introduction 
This section describes the computer package SIMLIN. 


SIMLIN is designed to solve n sets of equations in n unknowns. On your next 
terminal visit you will use this package, and the one described in the next unit. As 
you will have a lot of work to do on this visit, make certain that you go 
adequately prepared. Re-read the general instructions concerning computer 
packages given in Unit 1, and make quite certain before the terminal visit that you 
know the order in which to specify the options to solve a problem using SIMLIN. 
(An example is given in Subsection 4.2.) 
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4.2 A worked example 


Suppose we wish to use the computer package SIMLIN to solve the equations 


pao mee”. S, ace: 2x3 = 4 E, 
xX, —2x, + x3 = —4 E, 
Xi + Sx: + X3= 13 E, 


To do this, only a subset of the options described in Subsection 4.3 is required. 
The following computer dialogue illustrates the options needed for this particular 
problem. The information you have to type in is underlined. The rest of the 
printing is supplied by the computer. 


When you have logged on, and obtained the computer package SIMLIN, the 
computer will ask you to specify an option. You could then proceed as follows: 


OPTION? 10 
HOW MANY EQUATIONS DO YOU WANT TO SOLVE? 3 
OPTION? 11 
TYPE IN THE MATRIX A, ONE ROW AT A TIME, SEPARATING THE 
ELEMENTS BY COMMAS. 
ROW 17 2,—1 2 
ROW 2? 1, -2, 1 
ROW 3? 1, 5, 1 
OPTION? 12 | 
TYPE IN THE ELEMENTS OF MATRIX B, SEPARATING THE 
ELEMENTS BY COMMAS. 
ELEMENTS? 4, —4, 13 
OPTION? 20 
GAUSSIAN ELIMINATION METHOD WITH PARTIAL PIVOTING 
OPTION? 42 
PRINT SOLUTION ONLY 
OPTION? SOLVE 
SOLUTION 
VARIABLE VALUE 
1 2.03571 
2 2.42857 
3 —1.17857 
OPTION? 


At this stage, you are free to run another problem. If you type STOP at this stage, 
you will be able to call another computer package, or to end your session. 


It is worth noting that the instruction to solve the problem (SOLVE) should only 
be given after all the data and the method of solution have been specified. If you 
ask for the problem to be solved before this stage, an error message will be 
printed, asking you for more information. 


In the following description, the options we used in the example above are marked 
with an asterisk. 


4.3 A list of options in SIMLIN 


The options in this package are described under the same general headings as 
those in RECREL (Unit 2). 


Command options 


The control options are essentially the same as those in RECREL. Unlike the rest 
of the options described in this section, the command options have names rather 
than numbers associated with them. 


OPTIONS -—this option prints the list of options available to you in this 
package. 
SOLVE* —this option computes the solution. It is the last option you use 


when solving a problem. Before this is used, all the data and the 
method of solution must have been given to the computer. If any 
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of this information is missing, the computer will print an error 
message. 


HELP —help may be obtained if you need it, by typing HELP at any stage 
when a question mark appears. If you type HELP after the 
computer has printed OPTION?, you will be asked questions to 
help you to decide what option to use next. These questions 
require an answer YES or NO. If you type HELP while using an 
option, you will be given help as to how to use this option 
properly. 

LIST —this option prints out the information held in the computer about 
the current problem. It is advisable to ask for LIST before you 
ask for the probem to be solved. In this way, you can check that 
the computer is holding the correct information. 


STOP —this option is your means of exit from the package. You would 
only use this option if 
(i) you wanted to use another package, or 
(ii) you wanted to log-off the computer. 


ANSWER ~~ —this gives part answers to the end of section exercises so that you 
can check that you are on the right lines. To the question 


EXERCISE? 
you respond with the number of the exercise you are interested in. 


The rest of the options in this package are obtained by typing a number when 
the computer types OPTION? 


Problem options 
10* Specify the number of equations 
You will use option 10 to specify how many equations you want to solve. (You 


are limited to a maximum of fifteen equations.) This option must be used before 
you try to give the computer the data of the problem (options 11 and 12). 


11* Enter matrix A 

You will use option 11 to put the matrix A (the left-hand sides of the equations) 

into the computer, after you have specified the number of equations you want to 

solve by using option 10. This is done one row at a time. When the message 
ROW 1? 

appears, type in all the elements in the first row, separating the elements by 

commas. If you type too few elements an error message and an instruction to re- 


type the whole row will be printed. If you type too many elements, you will be 
given a warning message. 


12* Enter matrix b 
You will use option 12 to put the right-hand sides of the equations into the 
computer. When the message 

ELEMENTS? 


appears, type in all the right-hand side elements on the same line, separating them 
by commas. The error and warning messages are the same as in option 11. 


13 Edit A 


You can use option 13 if you wish to change a few individual elements in A. The 
two obvious circumstances in which you would want to do this are 


(i) if you made an error in typing in the elements, and 
(ii) if you were testing a problem for ill-conditioning. 


Suppose you wanted to change the element in the second row and third column of 
A to 8, and no other change. The computer dialogue would go as follows: 


OPTION? 13 
CHANGE ELEMENT IN ROW? 2 
AND COLUMN? 3 
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TO HAVE THE VALUE? 8 
DO YOU WANT TO CHANGE ANOTHER ELEMENT IN A? NO 
OPTION? 


Had you answered YES to the last question, you would have been asked to specify 
the next element you wanted to change. 


14 Edit b 


You can use option 14 if you wish to change any of the elements of b (the right- 
hand sides of the equations). The reasons you would wish to do this, and the 
method for doing it, are similar to option 13. Suppose, for instance, you wished to 
change the right-hand side of the third equation to —7.1. Then the dialogue with 
the computer would go as follows: 


OPTION? 14 
CHANGE ELEMENT IN ROW? 3 

TO HAVE THE VALUE? —7.1 

DO YOU WANT TO CHANGE ANOTHER ELEMENT IN B? NO 
OPTION? 


15 Choose a standard problem 


To save you having to type in some of the larger problems at the end of this 
section, and in your tutor-marked assignment, there is a set of problems already 
stored in the computer. Each stored problem has a name. For instance, Exercise 4 
at the end of this section is called PROB1. The following dialogue with the 
computer will access this problem. 


OPTION? 15 
PROBLEM NAME? PROB1 
OPTION? LIST 


It is advisable to print out the standard problem (using LIST) to check that the 
data is what you expected. 

16 Specify number of significant figures 

You can use this option if you want each stage of the calculation done to a fixed 
number of significant figures. For instance, the dialogue 


OPTION? 16 
HOW MANY SIGNIFICANT FIGURES DO YOU WANT 
TO USE IN THE CALCULATION? 3 


will cause each stage of the calculation to be done to three significant figures. 


You can specify any number of significant figures between 1 and 8. If you do not 
specify how many significant figures you wish to work to, the computer will work 
to its normal accuracy. 


Method options 

These options give you the choice of which variation of Gaussian elimination you 
wish to use. 

20* Partial pivoting 

If you choose option 20, the Gaussian elimination process will be done with 
partial pivoting. 

21 User row interchange 


This option allows you to choose which row you would like to use as the pivot 
row at each stage of the elimination. This option should be used in conjunction 
with the full printout option (option 40), otherwise you won’t know which row to 
use as a pivot row. 


22 Essential row interchange 


If you choose option 22, then Gaussian elimination will be done without pivoting, 
unless a zero pivot element is encountered. In this case, an essential row 
interchange will be made. 
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Print options 


These options give you the choice of how much detail you get printed out in the 
solutions. | 


40 Full printout 


This option gives the solution, a printout of each stage of the elimination, and the 
multipliers used to obtain it. It is interesting to see, but don’t ask for full printout 
too often—it takes up a lot of time! 


41 Outline printout 
This option gives the solution and a printout of the final stage of the elimination. 


42* Print solution only 
This option gives the solution only (no working on the way). 


A note of advice 


Once you have specified the options necessary to run your problem, those options 
will stay in the state that you set them, unless you specifically change them. This 
can be an advantage or a disadvantage. For instance, if you have just solved three 
equations in three unknowns and you want to solve another set of three 
simultaneous equations, you would only have to re-specify options 11 and 12 
(enter A and b)—as the computer retains the rest of the information from the last 
problem. However, if you have just done a problem using three significant figure 
arithmetic and subsequently wish to do a problem to the full accuracy of the 
computer, you must remember to re-specify the number of significant figures 
(using Option 16). 


Summary of Section 4 


This section describes the options in the computer package SIMLIN. This package 
solves sets of n simultaneous equations in n unknowns (n < 15). A standard 
sequence of options for solving a given system of equations is: 


10 Specify the number of equations 
11 Enter matrix A 
12 Enter matrix b 


20 Partial pivoting (method) 
42 Solution only 
LIST (optional—but informative) 


End of section exercises 


Exercise 1 
Use the computer package SIMLIN to solve the set of simultaneous equations 


Sx, = Ales $e = AL E 
—41x, + 5.2X + 3x3 = 4.1 E, 
SK + 3x5 eae 4.1x3 = 4.1 E; 


Exercise 2 
Use the computer package SIMLIN to solve the set of simultaneous equations whose 
matrices are 
4 -3 a 
Aw=11 0 6}, b= |-1}. 
4 -1 -2 0 
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Exercise 3 


If you have not already done the end of section exercise in Section 3, do it now, using the 
computer package SIMLIN. 


Exercise 4 
(i) Solve the set of simultaneous equations whose matrices are 


1 Lao ee 03 1 
O73 4 OS o>. 45 1 
A=|06 075 05- 043 1 5 = ET. 
US De = eas 057-075 1 
O45 5 O37 0.33.06 1 


This problem is already stored in the computer, to save you having to type in the 
coefficients, and is called PROB1. 
To obtain PROB(1, use option 15. When the program types 
PROBLEM NAME? 
you should type 
PROB1 


(ii) Is the solution you have found sensitive to small changes in the data? 
(Use option 13 to edit the data.) 


Exercise 5 


Use: (i) Gaussian elimination with essential row interchange, 
(ii) Gaussian elimination with partial pivoting, to solve the set of simultaneous 
equations whose matrices are 


See ee 11.4 
1) AS i793 

oe ee a ee : 16.6 
a eS 14.2 

ee SESE Se eee —11.7 

A= 2 ee OE oe ed , b=| —39 
Se ae ¢ See eee —4.8 

ee ee bE ee ae) 

i eee aes fee Fat 

4 8.9 


where the blank entries in A represent zeros. To save time, this problem has been stored in 
SIMLIN as PROB2 and can be retrieved using Option 15. 


Exercise 6 


(i) Solve the set of simultaneous equations whose matrices are 


0.45 0.1 0.16 0.09 0.04 —0.54 1.032 
0.47 0.183 0 0.18 —0.31 —0.36 0.3656 
<a 0.09 0.154 0.653 0.15 0.19 0.36 ae 1.6286 
OS 0.3 0.2 03 —02 Sa a 0.74 
—0.17 -—0.177 —0.163 —0.18 0.32 0.08 — 0.3794 
0.29 0.053 —0.762 0.06 —0.41 1.26 — 2.4176 


This problem is stored in SIMLIN as PROBS. 


(ii) Are the equations ill-conditioned ? 
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5 Special cases 


5.1 Homogeneous sets of equations 


Sometimes the equations we want to solve are of a special form, where the right- 
hand sides are all zero: 

a,X, + b,x, + ¢,x, =0 

- A,X, + b,x, + ¢,x, =0 

a,X, + b,x, + ¢3x, =0 
Such a system is said to be homogeneous. The principles developed already show 
how to deal with this type of equation. 
By inspection of the equations above, it can be seen that 

xX, =X,=x,=0 


is always a solution. An ‘all-zero’ solution of this kind is often referred to as the 
trivial solution. The question is, do sets of homogeneous equations ever have any 
solution other than the trivial one? As with the other examples in this unit, the 
elimination process will answer this question. 


Example 1 
Solve the simultaneous equations 
iy = Se ae SO E, 
X, + 2x, + 3x, =0 gE 
2X, + 3x,— x,=0 E, 
Proceeding as usual, we get 
Stage 1(a) Stage 1(b) 
1 1 1 O'R, 1 1 1 O;R, 
1|0 1 2 0|R,, 0 1 2 0/R,, 
2|0 1 —3 O/R,, 1L0 0 —5 0|R;, 


In this case, the back substitution only gives the trivial solution 
Xx, =X, =x, = 0. 


However, it is possible with some sets of homogeneous equations to obtain 
solutions other than the trivial solution, as the following example will demonstrate. 


Example 2 
Solve the simultaneous equations 

x, +X, 4+ X4 = 0 E, 

x, + 2x, + 3x, =0 E. 

2x, + 3x, + 4x; =0 ioe 
The elimination gives us 
Stage 1(a) Stage 1(b) 
1 1 1 O;R, 1 1 1 O;R, 

1) 0 1 2 0;R,, 0 1 Z 0/R,, 
2(0 1 Z 0|R,, 1| 0 0 0 0|R;, 


The row of zeros in Stage 1(b) represents an ‘equation’ 0 = 0. This ‘loss’ of an 
equation during the elimination process means that we can obtain solutions other 
than the trivial solution. This is because the equations we are left with, 
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X,+xX,+ x,=0 E, 
X, + 2x, =0 ew 
have an infinite number of solutions, as we shall now show. 
Let X,= *'&, 
then from E,,, x, = —2k 
ai rome xe 
So this set of equations has an infinite number of solutions of the form 
=k ts 2 Xe == kK, 
where k is arbitrary. 


In this last example, the reason why we obtained an infinite number of solutions 
was that the rows of A|b were linearly dependent. In Example 1, the rows of Alb 
were linearly independent, and as a result, we obtained only the trivial solution. 
We thus arrived at the following conclusion. 


If a set of n homogeneous simultaneous equations in n unknowns 
are linearly dependent, then they have an infinite number of 


solutions. If not, the equations only have the trivial solution 
y= Xo = ss See 


Exercise 1 
Solve the following sets of simultaneous equations. 


5X, + X,+ x,=0 E, 5x, — 18x, + 13x, =0 E, 


(Solution on p. 56) 


It is worth noting in the examples and exercises above, that in those homogeneous 
problems in which there were an infinite number of solutions: 


(a) the solutions were in proportion to each other, and 


(b) the trivial solution was always one of these solutions (obtained by putting 
Ks). 


To conclude this subsection, we look at an interesting connection between a set of 


“ simultaneous equations, and the corresponding set of homogeneous equations. 


Example 3 
If we were given the set of equations 


My + Ns + Xe=3 
Xie 25 +. 3%, =6 | (1) 
2%, + 3X, + 4x, =9 
theoretically we could solve them as follows: 
(a) First solve the corresponding set of homogeneous equations 
X, + X,+ x,=0 
X%; + 2h, + 3x, = 0 (2) 
2X; + 3x, + 4x, = 0 
As we saw, in Example 2, these equations have the solution 
Ma Sk, X, = —2k, x, =k, 


where k is arbitrary. 
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(b) Now try to spot a particular solution of the original equations (1). In this 
case, for instance, we can see that x; = X, = x3 = 1 1s a solution to these 
equations. 


From this information, we can now construct the general solution to the set of 
equations (2), as follows. 


As for recurrence relations, and differential equations, the general solution to a set 
of equations can be formed by adding a particular solution of these to the solution 
of the corresponding homogeneous set. Hence in this case, we can write down the 

solution 


x = 14k x, = 1 — 2k, Pee aS 


Although this was a useful technique in the case of recurrence relations and 
differential equations, and actually works for linear simultaneous equations, it 
doesn’t really help us much in the latter case. Most of the sets of equations we 
meet have unique solutions, and if we could spot the solutions to these, we would 
have no need for Gaussian elimination—and the problem would be done anyway! 


In the unit on eigenvalues you will meet a situation in which homogeneous sets of 
equations arise naturally. 


5.2 m equations in 7 unknowns (m = n) 


This unit has been restricted to discussing a method of solving n equations in n 
unknowns. Real life is not always kind enough to provide us with this ideal 
situation! In fact, when faced with m equations in n unknowns, where m # n, the 
techniques used are rather different from the method described in this unit. 


(a) More unknowns than equations 
This is a situation which we already know provides us with an infinite number of 
solutions (as long as the equations are not inconsistent). For instance, if we met 
the equations 
Xp Xo te SHS oe 
X,+Xx3,=2 E; 


this is a situation very similar to the one where we lost an equation after the 
elimination was complete. If we let x, = k, where k is arbitrary, we can write down 
the solution of the equations in terms of k: 


from E,, x, =2—-k, and from E,, x, = 1, giving a solution 
x =, x, = 2 — kK, Xs =k 
where k is arbitrary. 


Exercise 2 | 

Solve the simultaneous equations 
X, + 2x, + 3x3, =4 
X,— X,+ X,=6 


(Solution on p. 57) 


In general, if there are more unknowns than equations, there will be an infinite 
number of solutions (except in the rare case of inconsistent equations, in which 
case there will be none). 


(b) More equations than unknowns 
If there are more equations than unknowns, then the equations are normally 
inconsistent. For example, the following system of three equations in two 
unknowns 
X,+ X,=2 Ey 
Xx, + 2x, =3 = ae 
2X, + 3x, = 6 
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has no solution. The only values of x, and x, satisfying the first two equations are 
X,; = 1, x, = 1; but these values do not satisfy the third equation, as you can verify 
for yourself. 


However, if the third equation were changed to 


then the values x, = 1, x, = 1 would satisfy all three equations. The reason why 
this is now possible is that the three equations are linearly dependent; in fact we 
have E; = E, + E,, so that equation E, gives no new information that is not 
already in E, or E,. This possibility is, however, exceptional. During the Gaussian 
elimination process it would reveal itself by one of the equations reducing to 0 = 0 
at a certain stage. 


Summary of Section 5 


1. If all the right-hand sides of a set of simultaneous equations are zero (i.e. 
b = 0), the equations are called homogeneous. 


2. A set of n homogeneous equations in n unknowns will have: 
(i) an infinite number of solutions if the rows of Alb are linearly dependent: 


(ii) only the all-zero solution (called the trivial solution) if the rows of Alb are 
linearly independent. 


3. A set of m equations in n unknowns will have: 


(i) an infinite number of solutions if m < n (unless the equations are inconsistent, 
in which case there are no solutions); 


(ii) no solutions if m > n (unless enough of the equations are linearly dependent 
for the system to reduce effectively to n equations or less). 


6 End of unit test 


Section A 


These are multiple choice questions to which there is only one correct answer. Spend very 
little time on these. If you can’t do them, or get the answers wrong, re-read the appropriate 
section(s) in the unit. 


1. In one of the following, the rows of the matrix are linearly dependent. Which one? 
(a) [2 3 5 (b) {2 4 Si. 4ey 12 4 OL 4G} 42 4 6 

ae ees og ey 
2. Which of the following sets of simultaneous equations have no solution? 
(a) x + 3x, 5 (b) 2x,+4x,=0 (c) 2x, +4x,=0 

X,+ xX,=6 X,+ x,=0 Xe Xs = 0 

ee, a ea, 

(d) 2x, +4x,=8 (e) 2x; + 3x, =5 

x, +2x,=4 % + $x, = 6 


3. In which of the following sets of simultaneous equations would you expect unreliable 
solutions? 


(a) 3.0x, +2.4x, = 5.4 (b) 3.0x, +2.4x,= 5.4 (c) 3.0x, +24x,= 5.4 
2.2X, + 4.8x, = 7.6 5.9x, + 4.8x, = 10.8 9.8x, — 7.0x, = 16.2 

4. Choose the option which gives a true statement. Partial pivoting has the effect of: 

(a) making sure that all multipliers are less than 1; 

(b) eliminating ill-conditioning; 

(c) reducing the chance of error build-up; 


(d) eradicating the effects of rounding error. 
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Section B 


These are longer questions. 


5. Determine the nature of the solutions of the following sets of equations. In the case 
where the equations have a unique solution, find it. In the case where the equations have an 
infinite number of solutions: 


(a) find the general solution; 

(b) find which linear combination of the rows of A|b gives a row of zeros. 

(i) Xo Xe bee 7 a eS gS OS ee 
2x, + 3x, —x3; = —6 2h, Shy X= 8 2x; + 3X.— X3=— 6 
11x, + 9x,-—x,=-2 3x, — 4x, — 2x, = —3 9x, + x,+3x,= 23 


6. Use Gaussian elimination with partial pivoting to solve the following equations. 
3.1x, — 45x, + 7.1x, = 23.77 
4.2x, + 1.2x, — 1.7x3 = 29.95 
1.5x, — 3.5x, + 3.1x; = 4.510 


Do your work as accurately as your calculator will allow, but give your answer to two 
significant figures. 


7. Find the solution(s) of the following sets of homogeneous simultaneous equations. 

(i) X,—- X,+x,=0 (ji) x,- xX,+ x,=0 (iii) x, + 2x, + 3x, =0 
2x, + 3x, —x3;=0 2x, +3x,-— x,=0 x, + 3x, — 5x, =0 
11x, + 9x, —x,=0 3x, — 4x, — 2x, =0 


[Hint: use the eliminations for Question 5(i) and (ii) to do the first two parts of this 
question. | 
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Appendix: Solutions to the exercises 


Solutions to the exercises in Section 1 


1. Stage 1: ‘eliminate’ x, by taking E, — 2E,, which gives 
7x, = —14 E., 


Stage 2: from E,, we find x, = —2, and substituting this 
into E, we get 


xX, +4=35. 

Hence x, = 1, giving the solution 
x, =1, Xx, = -2. 

Check 


LHS of E,: x, — 2x, = 1 — (2 x (—2)) =5=RHS. 


LHS of E,: 2x, + 3x, = (2 x 1) + (3 x (—2)) = —4 = RHS. 


2. Stage 1: using the rule stated, E, — 2E, gives us 
7x, — 3x, — 3(5x, + 2x,) = —1— € x 20) 

which, when simplified, gives 

—2x, = —29 E,, 


Stage 2: from E,, we find x, = 5, and substituting this into 
E, gives us 


3X, + 10 = 20. 
Hence x, = 2, giving the solution 


x, = 2, x, =5. 


3. Stage 1(a): E, — 3E, gives 
—3x, + 44x, =} E,, 


and E, — $E, gives 


10 62s 

Be a Se ae E3, 

Stage 1(b): E,, — (=JE,, gives 
66. ae 

gers aoe E35 


Stage 2: from E,, we find x, = 1, and substituting this into 
E,, gives us 

— 3X2 + 4 = 4p. 

Hence x, = —1. Now from E,, 
3x, -1-1=1. 

Hence x, = 1, so the solution is 


x, =1, 
4. [3 a | 8 
4 7 oe 


5. The matrix representing these equations is 


x, = —l, x3, =1. 


1 2 4/R, 
| —1 ; R, 
Stage 1: 
1 2 4'R, 
R, - — —7 pling 


Stage 2: writing out the equations represented by the new 
matrix, we get 


xX, +2x,= 4 E, 


From E,, we find x, = 1, and substituting this into E, we 
get 

xX, +2=4, 

Hence x, = 2, giving the solution 

x, = 2, X,=1.— 


6. The matrix representing these equations is 


3 1 —1 1|R, 
5 1 2 6/R, 
4 —2 —3 aI, 
Stage I(a): 

3 1 — | IR, 
R,-§R,/0 -2 4 | Bir, 
Roe ee Sk. 

Stage 1(b): 
3 1 — 1 NR, 
eee 7 [Roa 
R,,—(92R,,10 0 2 | —s2lR,, 


Stage 2: the rest of the solution is the same as in Exercise 3. 
7. 


(1) The matrix is 


1 —2 5|R, 
; 1 | R, 
Stage 1: 

a = 
R, - — 9 —18 JR, 
Stage 2: the equations represented by the new matrix are 
Me eh 5 E, 
9x, = —18 E.. 


giving the solution 
x, =1, X, = —2. 


(i1) The matrix is 


2 —3 16|R, 
: —4 ei 
Stage 1: 
2 —3 16 
R, — — 2 ae 


Stage 2: the equations represented by the new matrix are 
2X, > 3ks = 16 
Ix, = — 2 
giving the solution 
x, = 2 x, = —4. 


8. (i) The matrix is 
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Stage I(a): Stage 1(a): 
t. =3 1 11)R, 3 —2 5 10|R, 
R, — 2R,| 0 7 2 —19)R,, R, —4R,|0 § $ 29'R>, 
R, — 3R,L0 4 —10 —22JR,, R, —4R,/0 i —18 28 1R,, 
Stage 1(b): Stage 1(b): 
1 —3 1 11]R, 3 —2 5 10|R, 
0 a 2 —19iR,, 0 8 4 29'R,, 
R;, — FR ,,L0 0 aa — FIR, R3, — “gR,,| 0 0 =or% ae R3, 
Stage 2: the equations represented by the new matrix are Stage 2: the equations represented by the new matrix are 
X,—3x%,+ *%,= jl E, 3x; —2x,+ 5x,;= 10 Ey 
7x, + 2x, = —19 E,, of ee E. 
— Fx,= -F Exp — Tx, = — te Esp 
giving the solution giving the solution 
y= 1, X, = —3, x, =1. sc 3. x5 = 2, eel, 
(ii) The matrix is 
3 —2 5 10]R, 
1 Zz 3 10 |R, 
1 > —3 | 10JR, 
Solutions to the exercises in Section 2 2: 
= (i) Stage 1(a): 
2 —1 —1 8/R, 
R,-3R,/0 4 -—4 | —OIR,, 
R; — 5R, |0 $ -— 7 + = 2, 
Stage 1(b): 
4 —1 —1 8iR, 
0 fe —6/R,, 
R;, — 3R,,| 0 0 0 16/R;, 


From R,, we get 0 = 16, so the given equations are 
inconsistent and have no solution. 


(ii) Stage I(a): 


1 — 2 4 TIR, 
R,— 2R,|0 8 — 9 —17|R,, 
R;—R, |0 16 —18 — 34/R,, 
Stage 1(b): 
1 —2 + 7\R, 
0 8 —9 —17)R,, 
R,, — 2R,,| 0 0 0 0O;}R;, 
Stage 2: the equations represented by the new matrix are 
By 2k, 4x E, 
8x, — 9x, = —17 E.. 
Let x, = k. Then from £,,, 
2 —17+4+ 9k 
8 


and from E,, 


—17 + 9k 


m= 742 3 ) - a 


11 — 7k 
eae 

Thus there are an infinite number of solutions, the general 
solution being 
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i —17 + 9k 


x, = < Pe = <n 
where k is arbitrary. 
(iii) Stage 1(a): 

3 — 2 3 
R, — 2R,| 0 43 —7 
e216 
Stage 1(b): 

3 — 2 3 2]R, 
ee ae 3 (Roa 

R;, — #4R,,L0 0 8 | —$IRy 


Stage 2: the equations represented by the new matrix are 


Be oe a 8 

3X, —7X3= 3 
oes 
bo See | 


giving the unique solution 

x, =1, Xx, = —l, xX, = —1. 

3. In Exercise 2(ii) we found 

R;, = R;, — 2R2, (1) 
where 

R,, = R, — 2R, and R,, = R, — R,. 
Substituting these expressions into (1) gives 
R,, = 3R, — 2R, + R;. 

Hence, 

3R, — 2R, + R, = 0. 

4. Stage 1(a): 


1 — 2 5 7|R, 

R, —R,| 0 5 — 9 Sid oe 
R, —R,| 0 20 — 36 33} R,. 
Stage 1(b): 

1 —2 5 TIR, 

0 5 —9 ie. 
R,, — 4R,,|0 0 0 —19)R,, 
From R,, we get 0 = —19, so the given equations are. 


inconsistent and have no solution. 
5. Stage I(a): 


ee. 5 | 6lR, 
R, —R, |0 ——e 
R, —2R,|0 i - 27 oe 


Solutions to the exercises in Section 3 


1. No solution given. 
2. Using three figure arithmetic, 


(0.2745 x 73.841) — 6.1879 
~ (0.275 x 73.8) — 6.19 
~ 20.3 — 6.19 
~ 14.1. 


This compares well with the exact answer, 14.081 455. 
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Stage 1(b): 

1 —2 5 6iR, 

0 5 —9 1|R,, 
R,, ee TR, 0 0 = — R;, 
Back substitution gives the unique solution 
x, =5, X= 2, x, = 1. 
6. Stage l(a): 

5 — | — | 2|R, 
R,-3R,)0 —-—#? $ S [Roa 
R, — §R,|0 3 =e —27 |R3 
Stage 1(b): 

5 — | —1 2/R, 
ee Se 
R,, + 4R,,| 0 0 0 0}R;, 


(i) As R,, = 0, there will be an infinite number of solutions, 


given by the equations 

a ae ee E, 
— 42x, + 3x; = F Ex, 

Let x, =k. Then from E,,, 
6k — 68 

eee 

and from E,, 

1 6k — 68 

p+ = 


00 


+ | 

5k — 6 

ae 

Thus the general solution is 

5k — 6 6k — 68 
eee 
where k is arbitrary. 
(11) From Stage 1(b), 
R;, + 4R,, = 0 


where from Stage 1(a), 


xX, =k 


R,, =R, — 2R, andR,, = R, — §R,. 
Substituting these expressions into (1) gives 
—2R,+4R, +R, =0. 


3. The matrix representing the equation is 


ee oe eo ee 
S29 ho eee ee. 
Gof ee ss ee 
oe eee ee ae 
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Stage I(a): 
1 -1l 1 -1l ~ R, 
2 0 0 1 4 |}-3 R,, 
0 1 —-2 -3 0 R;, 
0 0 -l 4 1 4 R,, 


As the second pivot is zero, change R,, with R;, (or with 
R,,): 


1 -1 1 -1 + R, 
0 et ee 0 R>, 
0 0 1 4|-3 ae 
1 a 1 4 Ry, 


Now we can proceed with the next stage of the elimination. 


Stage 1(b): 


ooo 


—1 
Stage 1(c): 


2 0 0 0 —-—10 10 + Ry 
Stage 2: back substitution gives the solution 
DS oe ie go 1. 


xy = 4, x, = -l, 


4. The matrix representing the equations is 


1.62 1.10 0.65 hs Bi R, 
6.18 4.20 —3.04 7.34 R, 
465 —3.05 2.10 3.70 R; 


(i) Using four-figure arithmetic, and no partial pivoting: 


Stage 1(a): 
1.62 1.1 0.65 Sat R, 
3.815 0 0.003 —5.52 | —5.52 R,, 
2.870 0 —6.207 0.234 | —5.972 R;, 
Stage 1(b): 
1.62044 0.65 a3) 4 
0 0003 <5.92..).—5.22. pas 
— 2069} 0 0 — 11420 | —11430|R;, 
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Stage 2: from R3, we find 
x3 = 1.001 (to four significant figures). 


Substituting this into the equation described by R2, gives us 
x, = 2.000 (to four significant figures). 


Finally from R, we get 

x, = 0.3206 (to four significant figures). 
Thus the solution we obtain by this method, 
x, = 0.3206, No = 2000, x, = LOut, 
is nothing like the correct solution. 


(ii) Again using four-figure arithmetic, but this time using 
partial pivoting, first we interchange R, and R;: 


6.18 4.20 —3.04 7.34 R, 7] 
1.62 1.10 0.65 3.37 R, 
465. —3.05 2.10 3.70 R, 


Stage I(a): 
6.18 420 —3.04 7.34 R, 
0.2621 0 —0.001 1.447 1.446 | R,, 
0.7524 p= 671 A387 | 1523-428 


Now R,,, and R;, must be interchanged: 


6.18 4.20 —3.04 7.34 R, 


ee Cee ev eee ek Bat) 
S -=000r- "14a a 


Stage 1(b): 
6.18 420 —3.04 7.34 R 
0 -—6.21 4.387 | —1.823 R,, 
0.0002 0 0 1.446 1.446 R3, 


Stage 2: proceeding exactly as in part (1), this time we obtain 
x; = 1,000, X= 1 Se ie 


which is actually correct (to four significant figures, of 
course!) 


Solutions to the exercises in Section 5 


1. 
(i) Stage I(a): 

2 3 — 5 OR, 
lc FF Ok, 
jlo | ORs, 
Stage 1(b): 

y 3 — 5 O;R, 

[See 3 *y | O1Ro, 
He Gk 


This gives only the trivial solution 


Xx, =X, =x, =0. 


(ii) Stage 1(a): 


Zz 3 — 5 O|R, 
a 3 2 | O}Ro, 
eo ee y | O}Rs, 
Stage 1(b): 

2 3 — 5 OR, 

eas zy | O}Ro, 
310 0 0 0) R;, 


As the rows are linearly dependent, we get an infinite number 
of solutions 


X, =X, =xX,=k 


where k is arbitrary. 
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2. The matrix is 


1 2 3 4iR, 
i —|] 1 i. 
Do one stage of the elimination: 
1 2 3 4iR, 
1 i —3 —2 A 
The equations represented by this matrix are 
X, + 2x, + 3x, =4 E, 
= 3x, — 2X, = 2 | 
Let x, =k. Then from E,,, 
— (2k + 2) 
x5 = ge 
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and from E,, 
—(2k +2 
ts be AG 
= ; 


Thus there are an infinite number of solutions, the general 
solution being 


—5k +16 = {2k +3) 
es ee 


where k is arbitrary. 


X,=k 


SS 


Solutions to the end of unit test 


Section A 


1. The answer is (c), because R, = 2R, in this matrix. 


2. The answer is (e). Multiplying E, of this set by 2 gives 
an equation which is obviously inconsistent with E,. 


3. The answer is (b). R, ~ 2R, in this matrix, so a small 
change in the initial data is likely to produce a large change 
in the solution. Another way of asking this question would 
have been: which set of equations is ill-conditioned? 


4. The answer is (c). Option (a) is wrong because it would 
allow a multiplier to be — 500, which would be most 
undesirable! Option (b) is wrong because no technique of 
solution can eliminate ill-conditioning. Option (d) is wrong 
because the effects of rounding error cannot be totally 
eradicated by partial pivoting—only reduced. 


Section B 
5. 
(i) Stage 1(a): 

1 — | 1 7IR, 
2) 0 5 — 3 —20 1... 
11}0 20 —12 —79iR,, 
Stage 1(b): 

1 —1 1 7IR, 

0 5 —3 —20/R,, 
4) 0 0 0 1/R,, 


From R,,, we see that these equations have no solution. 
(ii) Stage 1(a) 


1 —1 1 TIR, 
2/0 5 —3 —20|R., 
3} 0 —1 —5 —24)R,, 
Stage 1(b) 
1 —1] 1 7k, 
0 5 — 3 —20|R,, 
—3[0 ee eet 


Back substitution gives the unique solution 


x, =1, xX, = —1, x3 = 5. 


(1) Stage I(a): 


1 — 1 1 TIR, 
2/0 5 —3 —20)R,, 
910 10 —6 —40|R,, 
Stage 1(b): 

1 —|] 1 TIR, 

0 5 —3 —20)R,, 
2|0 0 0 0|R;, 


These equations have an infinite number of solutions. 


(a) Letting x, =k, back substitution gives the general 
solution 


ee eee Sh 2D 
ee ee 
where k is arbitrary. 

(b) From Stage 1(b), 

R;, — 2R,, =0 (1) 
where from Stage 1(a), 

R,, = R, — 2R, and R,, = R, — 9R,. 

Substituting these expressions into (1) gives 

—5R, — 2R, + R, = 0. 


6. As calculators differ in the degree of precision to which 
they work, the following numbers are just supposed to act as 
guide lines. 


xy 


X3 =k 


The largest entry in the first column is in row 2, so we 
exchange rows 1 and 2 to obtain the matrix 


4.2 i —1.7 29.95 |R, 
3.1 —4.5 tA 21) 1h A 
LS — 3.5 34 4.510/R, 
Stage I(a): 
4.2 LZ —1.7 29.95/R, 
0.738) 0 — 5.39 8.35 1.66/R,, 
0.357) 0 — 3.93 3.71 =~ 519iR 
Stage 1(b): 
4.2 t2 —1.7 29.95 IR, 
0 — 5.39 8.35 1.66 |R,, 
0.72910 0 — 2.39 — 7.40 /R3, 
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Back substitution now gives, to two significant figures, the 
solution 


%, = 71, x, = 45, x, = 3.1. 


7. (i) From Question 5(i), the matrix obtained after the 
elimination is 


1 ae 1 0 
0 2 = 3 0 
0 0 0 0 


Letting x, = k, back substitution gives the general solution 
2k XT 2 kK SH 
where k is arbitrary. 


(ii) From Question S(ii), the matrix obtained after the 
elimination is 


{ot t +-0 
0 {eS 
0 iy 22 7 6 


From this, we know that the equations have only the trivial 
solution 


Xx, =x, =x; =0. 


(iii) As we have only two equations in three unknowns, if a 
solution exists it will not be unique. Furthermore, as the 
equations are homogeneous we know that a solution does 
exist, namely the trivial solution. 


To find the general solution, do one stage of elimination. 
This produces the matrix 


1 ae | OjR, 
Letting x, = k, back substitution gives the general solution 


x, = —19k, %4 == Bk, x,=k 


where k is arbitrary. 
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